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DOCUMENT-IDENTIFIER: US 6058248 A 

TITLE: Computerized method for improving data resolution 



Abstract Text (1) : 

A data resolution synthesis algorithm takes low resolution source input data (e.g., 
data degraded from compression or data acquired from low quality imaging devices) 
to synthesize high resolution output data. In an exemplary embodiment for color 
printing, the algorithm is performed by characterizing a multi-pixel area, or 
window, around a pixel that can benefit from resolution enhancement. To interpolate 
to a high resolution output a set of spatial filters is applied to the data area 
based on the window characterization. The output of the resolution synthesizer is a 
set of multiple pixels for each input pixel, representing the source input pixel in 
a higher resolution enhanced version. The filters are chosen from a stored data 
base (generic or specifically applicable data base for each type input device) 
created to fit input/output device requirements. 

Brief Summary Text (10) : 

In color printing, such as with laser or ink- jet dot matrix hard copy apparatus, it 
may be even simply a user preference to create a print with higher resolution than 
the source provides in order to provide a sharper, less grainy, smoother textured 
image to present an overall higher quality and artistic print. 

Drawing Description Text (9) : 

FIG. 6 is a color comparison of images printed without and with the application of 
the present invention as shown in FIGS. 1-5. 

Detailed Description Text (5) : 

FIG. 1A is a depiction of a system 100 implementing the present invention. A data 
source 102 has an output of low resolution data 104. Before putting the data 104 
into an output device 106 (e.g., a color printer ) storage apparatus 108 from which 
an output version 110 of the input data 104 can be generated, the data 104 is 
analyzed. When the input data 104 is found to be congruent with the processes of 
the present invention, further analysis, data enhancement, and resolution synthesis 
steps 112 can be performed. In the process of analysis, other known manner data 
enhancements can also be employed to conform the output data 114 to the output 
device 106 data resolution capabilities, such as a variety of user output options 
provided in a color printer . 

Detailed Description Text (12): 

If the user has selected 107 BEST print mode output, but the computer has not 
requested a resolution upscalinq 109, a check is made to determine if the source 
data was already upscaled 111, e.g., the decompressed JPEG file. For example, 
decompression may have already upscaled the true source data by a factor of two or 
more via a simple pixel replication routine (see Background of the Invention 
section, supra) . Alternatively, the source may have stretched the data 
bidimensionally to fit the output device format [e.g., the Internet provider may 
know that pixel data needs to be upscaled in order to render a 6 . times . 9-inch print 
of the current screen display rather than a 2 . times . 3-inch print on the 
8 . 5 . times . 11-inch printer paper]. However, the algorithm used by the application to 
pre-upscale images is likely a simple and less effective prior art technique. Since 
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a goal of the present invention is to provide superior upscaling to those 
techniques while providing an output device resolution matched data set, reversal 
of such a prior upscaling, or downscaling 113, prior to further analysis and 
resolution synthesis 121 provides superior final image results. 

Detailed Description Text (20) : 

For a two-times ("2. times.") enhancement— viz . , two dimensionally from one input 
pixel to four output pixels — selection 203 (FIG. 2A) of a window 301 (FIG. 3) of 
3. times. 3 pixels has been found to present an adequate sample of a local region of 
an image to be enhanced. Note however that a larger window 305 (e.g., 5. times. 5 
pixels) could be used in the characterization, filtering, or both routines of the 
present invention. In experimenting specifically for ink-jet printing 
implementations, it has been found that use of a larger window both for 
characterizing a target pixel 303 and for filtering a window may not improve the 
printed image noticeably within the limitations of human visual perception. 
Selection of too large a window may lead to the printing of undesirable artifacts 
if the variations of image- characterizing factors is great within the selected 
window. However, such larger windows may be effective in another specific 
application (see Background section, supra) . Similarly, it is possible to select 
and use a "double window," one for characterization and another for data filtering. 
In the alternatives shown in FIG. 3, the outer 5. times. 5 pixel window 305 of the 
image that could be subject to filtering is denoted as observation vector z, while 
inner 3. times. 3 pixel window 301 is used to characterize the nature of the target 
pixel 303. Thus, the process is adaptable to optimization for different input data 
sources and output data devices . 

Detailed Description Text (27) : 

If the difference is above the threshold 206-208, indicating enhancement is 
warranted 206 f , 207', 208", the process continues on a plane-by-plane basis. Next, 
a check 211-213 is made of imaging characteristic factor variation for the entire 
window data subset select 203, again to determine if resolution synthesis is 
warranted. For example, while a target pixel may vary from the surrounding window 
as determined in steps 206, 207, 208, if the window 301 itself has little overall 
color variation — e.g., the image of the white of an eye — applying resolution 
synthesis could be superfluous to the visual appearance of the final output image 
and so it can be bypassed in order to speed data processing. In the preferred 
embodiment, a grey scale luminance value is calculated 211 for the target pixel 303 
and each pixel in the window 301. 

Detailed Description Text (28) : 

For the current preferred embodiment useful in computer peripheral color printing, 
it has been found that suitable results are achieved in a reasonable real time 
calculation by using grayscale values even though manipulating color data of a form 
comprising tristimulus color space triplets such as {Red, Green, Blue} or {Cyan, 
Magenta, Yellow} . Each data triplet value is run through the calculations 
individually as if it represented a grey scale data coordinate. In other words, 
each color plane data coordinate of a triplet is treated individually as if it were 
a single luminance value in grey scale: 

Detailed Description Text (29) : 

It will be recognized by a person skilled in the art that a data base using true 
color space data can be created and used to created tristimulus color space vectors 
and data filters but at the cost of adding significantly more memory, logic, and 
data processing time. As a variety of such techniques are already known in the art, 
use of true color space data rather than grey scale luminance values is not beyond 
the scope of the present invention, but no further detail is necessary to an 
understanding of the present invention. Note that selection of grey scale luminance 
makes the process suitable for true grey scale image enhancement, e.g., for 
resolution enhancement of black and white photographs. 
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Detailed Description Text (30) : 

The changes in luminance (Max L*-Min L* , where L* is the standard nomenclature for 
luminance ) , over the target pixel window of pixels 301 is calculated and compared 
212 to a predetermined threshold. The difference is compared to a first threshold 
of a range of luminance values. If it is above the first predetermined threshold, 
the process proceeds. If not above the first threshold, meaning very little 
variation in the window 301, another simple known in the art enhancement, e.g., 
pixel replication, or the like, is applied 209. With the process proceeding, the 
entire window luminance variation of the window is checked 213 against a second 
predetermined threshold. In other words, a test is made as to whether the average 
luminance of a window 301 is within a range of color variation—meaning some but 
not a great deal of variation. Thus, if the difference 213 is above the lower 
limit, first threshold, of the range of color variation 212 but below the upper 
limit, second threshold — meaning subtle color variations within the window — a 
secondary, improved enhancement such as a asymmetric Laplacian filter is applied 
215 to the target pixel. That is, within the luminance check range, resolution 
synthesis may not been found to be warranted, yet known manner enhancement is 
desired and therefore applied. If in comparison the window luminance variation 
exceeds the second threshold 213, the process proceeds. 

Detailed Description Text (33) : 

Turning now to FIGS. 2B and 3, assume a window 301, or at least one color plane of 
the current pixel 303, is selected for resolution synthesis. For the printing 
implementation of the exemplary embodiment of the present invention, the target 
pixel window region 301 is characterized 221, 309. The characterization process is 
depicted in more detail in FIG. 4. Each axis in the 8-dimensional space L0-L7 has a 
designated range of 0 to 255. The target pixel LT and eight surrounding pixels L0- 
L7 were converted to gray scale luminance values; now the central target pixel LT 
value is subtracted: 

Detailed Description Text (35) : 
Ln- luminance value for surround pixel n, 

Detailed Description Text (36) : 

LT= luminance value for the target pixel, and 

Detailed Description Text (37) : 
Vn=relative luminance values of LT to Ln. 

Detailed Description Text (38) : 

This provides an 8-dimensional vector denoted the cluster vector y (also sometimes 
referred to hereinafter as a "window key" or "cluster key"), which is a 
representation of the given pattern, tone independent, of the window data for the 
entire pixel window 301. In other words, a real time derived cluster vector y 
contains salient image data for the pixel window 301. Cluster vectors y are assumed 
to be distributed as a multi-variate Gaussian mixture via this luminance variation 
value characterization. 

Detailed Description Text (46) : 

In this exemplary embodiment, the provided data base is one hundred and one image 
resolution data filters (the number of filter classes is constrained only by 
storage capacity of the particular implementation) , each class comprising groups of 
four data filters that have been generated and classified based upon imagery test 
data. For the exemplary embodiment, each class is a group of four data filters 
relating to a luminance grey scale pattern. 

Detailed Description Text (66) : 

Despite the seeming complexity of the new enhanced high resolution image from low 
resolution image data, it has been found that the data processing time in the 
efficient resolution synthesis mode is negligible with respect to the average print 
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Brief Summary Text - BSTX (8): 

The shaded rendering process; in box 30, takes the transformed, clipped 
primitives for the objects in the selected view of the scene, simulates the 
illumination on and in the scene, according to a physically-based model, 
typically an approximation to the physics of light in the scene, in order to 
determine a pixehcolorfor every pixel \ of every primitive in the scene, and 
produces the image pixels a nd pixel colors needed to display the scene. 
Shading attempts to simulate how light in the scene behaves at the surfaces of 
the objects in the scene. "Ray tracing", generally defined as an image 
synthesis technique using geometrical optics and rays to recursively evaluate 
shading and visibility, and "radiosity"; generally defined as an image 
synthesis technique which balances the distribution of light energy in an 
environment ; are two well known shaded renderin g meth ods for i mage synthesi s . 
Steps in a shaded rendering process typically include first determining which 
objects (or surfaces of objects) in the scene are seen by the observer from a 
selected scene viewing position, determining the light arriving at each point 
on each visible object surface in the scene, simulating the propagation of that 
light in th e s cene forlhat "point, and assigning a color value to an image; l; 



: 



pixel as a function of the light reflected and transmitted at that point by the 
light sources and other objects in the scene. Once shaded rendering process 30 
is completed, the image to be displayed is represented as a collection of 
individual image pixel s J each with an associated pixel '- color. The image 
pixels; shown- in memory device 32, are typically stored in a frame buffer from 
which they may be processed, via arrow 34 to box 48, by display routines 
appropriate for the specific device on which the image is to be generated, for 
display or reproduction on an output medium 49 




■ v.. .... 



Brief Summary Text - 



>rs physically producible on a particularl col or output medium 



I CttGT IStM ACM nttlRta CB 

«t <U Calor Onl Nnfet Ovw 



. Jhtanmtato 6b nan. ftczatatni* 
BBdt niw<u hUM fa 11* 1*14* QnOMi) 
■ ftia urn ml wn M*a4 Mtoft, « ft» 



ante |*ifu 



■ nib 0E{gmLijral faith 

><tMB»il^ oaten * 1A A TOgi the MHtiJ 

4ubLl n. rMoa^ to*** 



1IST AVAILABLE COPY 



iBstartli a 0 m « m 4 



j;g]ln.. 



PIE-- 



SI 



S? 1 :58 PM 



EAST -[lumnochroma157.wsp:1] 



(BEAST Browser - L19: (3) (color$1 and... | US 51 1 1302 A | Tag: S,T1 1 Doc: 3/3 | Format : KWIC 



D Eile 



mam 



Edit 



Help 



View loots Window 

gBliJ Dial jgiNAh 



P] ^pP a | VUI i;|HllBl|BlN Kafei -U^l 



Ftfjdwhett: chroma 



Eind Next 



Area 

C y : 


I:--;:- 


i Direction - • 

cup 




- Match word fiigl % 
C Whole r leit 


* 


- Look in - - - 1 


/♦ Sei/Cur 




f fiown 




I ©"Part Right 








XJpiments of 
d to a 
oved 
>. In 
: itinql : 
Me 

lored 



dote-Wnted ih W supef pixel: IS limited to th6 huftlber of Supgivtaeg 
individual pixels therein. In this manner, dot-next-to-dot-always (DNDA) 
printing is assured, and there is never any dot-on-dot (DOD) color mixing and 
the above problems related to lack of toner transparencies. 
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Detailed Description Text - DETX(44): 

In another aspect or specie of the present Invention, either m on 
or color gray scale printinq ! mav be achieved by forming dots 
electrophotographically in a dot-next-to-dot (DND) fashion in a plurality of 
super pixels as described above so that the maximum number of dots printed in 
each super pixel never exceeds the number of individual or subdivided / pixels: 
therein. In addition, the area of each printed dot is varied within each pixel r 
areaby controlling the dot size printed in each pixel in a manner which is 
related to a desired level of gray scale to be achieved in the printed image 
The dot size may be controlled, for example, by controlling the pulse width or 
power of the laser beam (which translates into laser beam spot size) used 
within a color laser print engine of the type described above; 



Detailed Description Text - DETX (45): 
Referring now to FIG. 58, there are shown five super pixels 156, 158, 160, 
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i m scnem e o r ao i s assignment t o a zmrn^Ti supe mM ren ts a55> 3 n tn<5 
dots counts in the sequence of their highest gray level plane from black, cyan, 
magenta to yellow. This scheme is then repeated as described in the example 
given above for medium and low gray level 
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Detailed Description Text - DETX (33): 

Various modifications may be made in the above described embodiment without 
departing from the scope of this invention. As an example, a different dye 
loading ratio of 8:3:1 may be used with a 32 level gray scale and with a 
2. times 2 super pixel! in accordance with the above teachings. In addition, 
this invention may be used with different size drop volumes or with multi-drop 
formatting where tno print area is only a single oixel However , in the 
multi-drop printing process, higher print frequencies are required as compared 
to those required by the present invention Also, the present invention may be 
used with other types of ink jet printers such as thermal transfer or 
piezoelectric ink jet printers 
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Detailed Description Text - DETX (35): 

In priority of selecting the highest drop count from the (C, Y, M, K) 
colors, there may be an advantage to modify this set to only (CM, Y) colors 
only and to leave the black or K plane of information intact. This is because 
the black plane contains most of the resolution information, and this is 
especially true for the case of 100% UCR where the error diffusion of the color 
plane will only slightly alter the chromaticity of the printed image 
Additionally, the visual perception is more sensitive to resolution than it is 
to color information As a practical matter, the yellow gray scale is of least 
importance, so from both a hardware and a software standpoint, a single dye 
level of yellow would be acceptable. : 
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Detailed Description Text - DETX (5): 

With reference again to FIG. 1 ( at step 40, within the image, an area to be 
dodged is defined, through standard area definition techniques. This may be 
accomplished with a manual outlining arrangement, where the area is outlined 
with appropriate mouse software, or object/edge finding software, or any other 
area selection technique. The area to be dodged may be either inside or 
outside the area defined. When the process of dodging is selected, at step 4 
a new look up table is created having perhaps similar chrominance values/but 
different luminance; values representing the bri ghtness ! of theirnage. Again, 
with reference to U.S. patent application Ser. No. 07/517,895, by James E. . 
Boilman et al. t entitled "Color Editing With Simple Encoded Images", the 
manipulation of colors in the defined portion of the original image would be 
accomplished in the reduced color set representation of the original image, 
through the user interface I of FIG. 3, which shows a window arrangement where, 
below the image display space 41 , sliders 42, 43. 44 and 46, respectively 
representing hue, saturation, black and white contrast, and brightness, are 
selectable by movement of indicators 49 with a mouse or other data entry 
device. Hi jo; and saturation represent chrominance : values, and variations of hue 
and saturation vary the actual color of the image. Black and white contrast 
and :bnghtness represent :luminance; values, and variations of black and white-; 
contrast and brightness represent variations in the intensity of the image. 
While dodging refers to black and white contrast and brightness variation in 
photographic sciences, in the context of the present invention, dodging refers 
to variation of image appearance, coupled with softened edges, and may be used 
with hue, saturation, black and white contrast or brig htness variations, or 
combinations thereof Other variations are also within the scope of the 
invention. With reference to FIGS. 1 and 2, at step 50 in the dodging process, 
the image appearance characteristics, of the defined area, to be dodged are; 
varied, by adjusting the image data values in lu minance/chrominance space, 
Variations to the image look tip table m the luminance/chrominance color space : 
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